Lời giải:

Áp dụng BĐT Bunhiacopxky:

\((\sqrt{x^2+xyz}+\sqrt{y^2+xyz}+\sqrt{z^2+xyz})^2=(\sqrt{x}.\sqrt{x+yz}+\sqrt{y}.\sqrt{y+xz}+\sqrt{z}.\sqrt{z+xy})^2\)

\(\leq (x+y+z)(x+yz+y+xz+z+xy)=xy+yz+xz+1\)

\(\Rightarrow \sqrt{x^2+xyz}+\sqrt{y^2+xyz}+\sqrt{z^2+xyz}\leq \sqrt{xy+yz+xz+1}\)

\(\Rightarrow A\leq \sqrt{xy+yz+xz+1}+9\sqrt{xyz}\)

The BĐT AM-GM (Cô-si) thì:

\(1=x+y+z\geq 3\sqrt[3]{xyz}\Rightarrow xyz\leq \frac{1}{27}\)

\(x^2+y^2+z^2\geq xy+yz+xz\Rightarrow (x+y+z)^2\geq 3(xy+yz+xz)\)

\(\Rightarrow xy+yz+xz\leq \frac{1}{3}\)

\(\Rightarrow A\leq \sqrt{\frac{1}{3}+1}+9\sqrt{\frac{1}{27}}=\frac{5\sqrt{3}}{3}\)

Vậy \(A_{\max}=\frac{5\sqrt{3}}{3}\Leftrightarrow x=y=z=\frac{1}{3}\)

\(x+y+z=xyz\Rightarrow\dfrac{1}{xy}+\dfrac{1}{yz}+\dfrac{1}{zx}=1\)

Đặt \(\left(\dfrac{1}{x};\dfrac{1}{y};\dfrac{1}{z}\right)=\left(a;b;c\right)\Rightarrow ab+bc+ca=1\)

\(P=\dfrac{2a}{\sqrt{1+a^2}}+\dfrac{b}{\sqrt{1+b^2}}+\dfrac{c}{\sqrt{1+c^2}}=\dfrac{2a}{\sqrt{ab+bc+ca+a^2}}+\dfrac{b}{\sqrt{ab+bc+ca+b^2}}+\dfrac{c}{\sqrt{ab+bc+ca+c^2}}\)

\(P=\dfrac{2a}{\sqrt{\left(a+b\right)\left(a+c\right)}}+\dfrac{b}{\sqrt{\left(a+b\right)\left(b+c\right)}}+\dfrac{c}{\sqrt{\left(a+c\right)\left(b+c\right)}}\)

\(P=\sqrt{\dfrac{2a}{a+b}.\dfrac{2a}{a+c}}+\sqrt{\dfrac{2b}{a+b}.\dfrac{b}{2\left(b+c\right)}}+\sqrt{\dfrac{2c}{c+a}.\dfrac{c}{2\left(c+b\right)}}\)

\(P\le\dfrac{1}{2}\left(\dfrac{2a}{a+b}+\dfrac{2a}{a+c}+\dfrac{2b}{a+b}+\dfrac{b}{2\left(b+c\right)}+\dfrac{2c}{c+a}+\dfrac{c}{2\left(c+b\right)}\right)=\dfrac{9}{4}\)

\(P_{max}=\dfrac{9}{4}\) khi \(\left(a;b;c\right)=\left(\dfrac{7}{\sqrt{15}};\dfrac{1}{\sqrt{15}};\dfrac{1}{\sqrt{15}}\right)\) hay \(\left(x;y;z\right)=\left(\dfrac{\sqrt{15}}{7};\sqrt{15};\sqrt{15}\right)\)

Ta có: \(x+y+z=xyz\Rightarrow x=\frac{x+y+z}{yz}\Rightarrow x^2=\frac{x^2+xy+xz}{yz}\Rightarrow x^2+1=\frac{\left(x+y\right)\left(x+z\right)}{yz}\)\(\Rightarrow\sqrt{x^2+1}=\sqrt{\frac{\left(x+y\right)\left(x+z\right)}{yz}}\le\frac{\frac{x+y}{y}+\frac{x+z}{z}}{2}=1+\frac{x}{2}\left(\frac{1}{y}+\frac{1}{z}\right)\)\(\Rightarrow\frac{1+\sqrt{1+x^2}}{x}\le\frac{2+\frac{x}{2}\left(\frac{1}{y}+\frac{1}{z}\right)}{x}=\frac{2}{x}+\frac{1}{2}\left(\frac{1}{y}+\frac{1}{z}\right)\)

Tương tự: \(\frac{1+\sqrt{1+y^2}}{y}\le\frac{2}{y}+\frac{1}{2}\left(\frac{1}{z}+\frac{1}{x}\right)\); \(\frac{1+\sqrt{1+z^2}}{z}\le\frac{2}{z}+\frac{1}{2}\left(\frac{1}{x}+\frac{1}{y}\right)\)

Cộng theo vế ba bất đẳng thức trên, ta được: \(\frac{1+\sqrt{1+x^2}}{x}+\frac{1+\sqrt{1+y^2}}{y}+\frac{1+\sqrt{1+z^2}}{z}\le3\left(\frac{1}{x}+\frac{1}{y}+\frac{1}{z}\right)=3.\frac{xy+yz+zx}{xyz}\)\(\le3.\frac{\frac{\left(x+y+z\right)^2}{3}}{xyz}=\frac{\left(x+y+z\right)^2}{xyz}=\frac{\left(xyz\right)^2}{xyz}=xyz\)

Đẳng thức xảy ra khi \(x=y=z=\sqrt{3}\)

\(gt\Leftrightarrow\dfrac{1}{\sqrt{x}}+\dfrac{1}{\sqrt{y}}+\dfrac{1}{\sqrt{z}}=1\)

\(P=\dfrac{1}{xyz}\left(x\sqrt{2y^2+yz+2z^2}+y\sqrt{2x^2+xz+2z^2}+z\sqrt{2y^2+xy+2x^2}\right)\)

\(=\dfrac{1}{xyz}\left(x\sqrt{\dfrac{5}{4}\left(y+z\right)^2+\dfrac{3}{4}\left(y-z\right)^2}+y\sqrt{\dfrac{5}{4}\left(x+z\right)^2+\dfrac{3}{4}\left(x-z\right)^2}+z\sqrt{\dfrac{5}{4}\left(x+y\right)^2+\dfrac{3}{4}\left(x-y\right)^2}\right)\)

\(\ge\dfrac{1}{xyz}\left[x.\dfrac{\sqrt{5}\left(z+y\right)}{2}+y.\dfrac{\sqrt{5}\left(x+z\right)}{2}+z.\dfrac{\sqrt{5}\left(x+y\right)}{2}\right]\)

\(=\dfrac{\sqrt{5}\left(z+y\right)}{2yz}+\dfrac{\sqrt{5}\left(x+z\right)}{2xz}+\dfrac{\sqrt{5}\left(x+y\right)}{2xy}\)

\(=\dfrac{\sqrt{5}}{3}\left(1+1+1\right)\left(\dfrac{1}{x}+\dfrac{1}{y}+\dfrac{1}{z}\right)\ge\dfrac{\sqrt{5}}{3}\left(\dfrac{1}{\sqrt{x}}+\dfrac{1}{\sqrt{y}}+\dfrac{1}{\sqrt{z}}\right)^2=\dfrac{\sqrt{5}}{3}\) (bunhia)

Dấu = xảy ra khi \(x=y=z=9\)

 Thấy : \(\sqrt{2y^2+yz+2z^2}=\sqrt{\dfrac{5}{4}\left(y+z\right)^2+\dfrac{3}{4}\left(y-z\right)^2}\ge\dfrac{\sqrt{5}}{2}\left(y+z\right)>0\) 

CMTT : \(\sqrt{2x^2+xz+2z^2}\ge\dfrac{\sqrt{5}}{2}\left(x+z\right)\)  ; \(\sqrt{2y^2+xy+2x^2}\ge\dfrac{\sqrt{5}}{2}\left(x+y\right)\) 

Suy ra : \(P\ge\dfrac{1}{xyz}.\dfrac{\sqrt{5}}{2}\left[x\left(y+z\right)+y\left(x+z\right)+z\left(x+y\right)\right]\)

\(\Rightarrow P\ge\sqrt{5}\left(\dfrac{1}{x}+\dfrac{1}{y}+\dfrac{1}{z}\right)\) 

Ta có : \(\sqrt{xy}+\sqrt{yz}+\sqrt{xz}=\sqrt{xyz}\Leftrightarrow\dfrac{1}{\sqrt{x}}+\dfrac{1}{\sqrt{y}}+\dfrac{1}{\sqrt{z}}=1\) 

Mặt khác :   \(\dfrac{1}{x}+\dfrac{1}{y}+\dfrac{1}{z}\ge\dfrac{\left(\dfrac{1}{\sqrt{x}}+\dfrac{1}{\sqrt{y}}+\dfrac{1}{\sqrt{z}}\right)^2}{3}=\dfrac{1}{3}\)

Suy ra : \(P\ge\dfrac{\sqrt{5}}{3}\)

" = " \(\Leftrightarrow x=y=z=9\)